Intentional Differences and Improvements

This directory documents stable, user-visible differences and improvements between Provenant and the Python ScanCode Toolkit reference implementation. It includes parser improvements and text-detection subsystem improvements.

Philosophy

This directory records cases where Provenant intentionally differs from or improves on the Python reference implementation. We:

• Fix bugs present in the Python implementation
• Implement features marked as TODO in Python
• Add missing functionality where it improves user-visible behavior
• Improve data quality and extraction accuracy

Improvement Categories

🐛 Bug Fixes

Python implementation has incorrect behavior, we fix it.

✨ New Features

Python has TODO comments or placeholders, we implement the feature.

🔍 Enhanced Extraction

Python extracts some data, we extract more (additional fields, better parsing).

🛡️ Security Improvements

Python has unsafe patterns (code execution, DoS vulnerabilities), we use safe alternatives.

Summary Table

Area	Improvement Type	Python Reference Limitation	Rust Improvement	Impact
RPM	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Truncated installed EVR, extension-only source RPM detection, missing YumDB enrichment, and thin archive metadata	Full dependency extraction, EVR preservation, hash-based source RPM recognition, richer archive metadata, and YumDB merge support	Better RPM/rootfs parity, cleaner package identity, and richer system-package provenance
Debian	✨ New Feature + 🔍 Enhanced + 🐛 Bug Fix + ⚡ Performance	No direct .deb archive introspection, incomplete DEP-5 copyright detections, missing installed-package sidecar integration, and avoidable repeated paragraph parsing work	Direct .deb control/archive introspection, package-matching embedded copyright recovery, coherent DEP-5 primary/paragraph/fallback detections, installed Debian sidecar integration for rootfs/container scans, and a Debian-local single-pass paragraph finalizer	Better Debian archive metadata accuracy, improved Debian copyright visibility, stronger Debian rootfs/container package inventory, and faster large DEP-5 parsing
Conan	✨ New Feature	No conanfile.txt or conan.lock parser	Full conanfile.txt + conan.lock extraction	C/C++ dependency visibility
BitBake	✨ New Feature + 🔍 Enhanced	No shipped Python BitBake parser, no .bbappend packaged metadata coverage, and no parser-owned recovery of local LIC_FILES_CHKSUM / SRC_URI file references	Added bounded static .bb + .bbappend parsing, override-style dependency handling, raw declared-license preservation, local file-reference extraction, and sibling assembly for recipe-plus-append metadata	Better Yocto/OpenEmbedded package visibility and stronger recipe-local provenance
Carthage	✨ New Feature + 🔍 Enhanced	No shipped Python Carthage parser	Added static Cartfile, Cartfile.private, and Cartfile.resolved parsing with dependency origin preservation and sibling assembly for declared plus pinned state	Better Apple dependency visibility for Carthage-managed projects
CPAN	✨ New Feature + 🐛 Bug Fix	Stub-only handlers (no parse method), and malformed inputs in Rust fallback paths could lose parser identity	Full META.json, META.yml, MANIFEST parsing plus datasource-preserving fallback package identity for malformed/unreadable CPAN inputs	Perl metadata extraction with safer assembly/scanner behavior
RPM Specfile	✨ New Feature	Stub with TODO comment	Full preamble parsing	RPM spec metadata extraction
CPAN Makefile.PL	✨ New Feature + 🔍 Enhanced	Stub-only handler (no parse method)	WriteMakefile metadata extraction plus bounded VERSION_FROM and ABSTRACT_FROM recovery from sibling module files	Better Perl build metadata and package identity
OSGi Manifest	✨ New Feature	Empty path_patterns (assembly only)	Full OSGi metadata extraction	Java bundle dependencies
Gradle	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Wrong compileOnly runtime classification, missing Gradle SBOM license propagation, incomplete version-catalog alias identifiers, and only-first-block dependency extraction	Fixed scope classification, extracted Gradle POM license metadata, parsed all discovered Gradle dependency blocks, resolved TOML-backed libs.versions.toml aliases, and preserved nested project-path identifiers	Better Gradle dependency semantics, better SBOM metadata, and more accurate package identifiers
Gradle Module Metadata	✨ New Feature + 🔍 Enhanced	No merged .module metadata handler despite upstream attempts	Added direct Gradle .module parsing with published artifact checksums, dependency and constraint extraction, variant metadata preservation, and file-reference recovery	Better JVM publication metadata, stronger artifact provenance, and better coverage of modern Gradle-native modules
Gradle Lockfile	✨ New Feature	No gradle.lockfile parser	Full lockfile dependency extraction	Pinned dependency auditing
vcpkg	✨ New Feature + 🔍 Enhanced	No modern vcpkg.json manifest parser and no preservation of manifest-mode dependency/configuration metadata	Added strict-JSON vcpkg.json parsing for project and port manifests, direct dependency extraction, versioning metadata preservation, and embedded/sibling configuration capture	Better modern C/C++ dependency visibility and stronger manifest-mode provenance
Bun Lockfile	✨ New Feature + 🔍 Enhanced	No Bun lockfile parser, no Bun binary lockfile compatibility, and no Bun lockfile sibling/workspace assembly coverage	Added JSONC-aware bun.lock parsing, static legacy bun.lockb v2 compatibility, workspace-aware dependency extraction, and Bun sibling/workspace assembly support	Better Bun dependency visibility, safer legacy Bun compatibility, cleaner npm-family assembly, and stronger modern JS lockfile coverage
npm Workspace	✨ New Feature	NonAssemblable stub + basic assembly	Workspace extraction + improved assembly	Monorepo structure visibility + correct package counts
npm + Yarn	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Dropped package.json platform metadata, anonymous hard-failure fallback rows for malformed package.json, incomplete modern lockfile/root handling, lossy Yarn Berry non-@npm: resolution detail, no Yarn manifest-based scope inference, and package-root ownership gaps	Surfaced resolutions/overrides pins, package.json platform metadata, identified malformed-manifest fallback rows, richer Yarn Berry resolution metadata, fixed scoped npm fallback URLs and metadata normalization, modern lockfile/link handling, Yarn scope inference, and deterministic nested/workspace ownership	Better npm/Yarn parity, clearer malformed-manifest output, cleaner lockfile metadata, better manifest compatibility visibility, and correct bundled/workspace package attribution
npm Resolutions/Overrides	✨ New Feature + 🔍 Enhanced	Yarn resolutions forced to runtime/direct intent; npm overrides never surfaced as dependencies	Emit Yarn resolutions and npm overrides as distinctly scoped pin dependencies (is_pinned = true) with intent booleans left unset per the honest-unknowns guardrail	Honest pinned-dependency surface for npm/Yarn override blocks without asserting unproven intent
Structured metadata	✨ New Feature + 🔍 Enhanced	CITATION.cff and publiccode.yml were only tracked as planned metadata surfaces, not shipped parsers	Added bounded standalone parsers for CITATION.cff and publiccode.yml, including SPDX-aware declared-license normalization, party extraction, parser goldens, and datasource accounting	Better project provenance and descriptive metadata without evaluating ecosystem tooling
Hugging Face	✨ New Feature	No Hugging Face model/dataset metadata parser and no pkg:huggingface PURL emission	Added static parsers for model-card README.md frontmatter, Transformers config.json, and Diffusers model_index.json; emits pkg:huggingface/<ns>/<name> only from a checked-in _name_or_path/model_name, with SPDX-normalized declared license and base_model/datasets dependencies	Hugging Face model/dataset provenance with honest, no-guess identity
Composer	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Alternate file names unsupported, nested package files not assigned, and large lockfiles still too eager in memory behavior	Added alternate-name support, nested package resource assignment, lighter large-lock extraction, and richer manifest provenance/party typing	Better Composer package detection, better nested file attribution, safer large-lock handling, and richer manifest provenance
Conda	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Installed package files from conda-meta not assigned, ambiguity between symbolic channel namespace and channel URL prefixes, and parser/assembly/docs drift around real environment filename aliases	Added conda-meta file assignment via merged rootfs metadata, explicit channel-vs-channel_url disambiguation, and consistent .yml / .yaml / hyphenated environment alias handling across parsing, assembly, and supported-format metadata	Better Conda rootfs package assignment, cleaner package identity/origin semantics, and more reliable environment-file detection
Alpine	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Missing APKBUILD recipe parsing, no APKBUILD dependency extraction, uneven license normalization across Alpine surfaces, incorrect raw matched text for custom:multiple, fileless package confusion, and no HTTPS Alpine VCS URL	Added static APKBUILD parsing with dependency families, preserved raw custom:multiple matched text, normalized trustworthy licenses across APKBUILD / installed-db / .apk, retained fileless packages, and emitted HTTPS Alpine commit vcs_url	Better Alpine recipe parity, safer package evidence, cleaner Alpine provenance, and more consistent declared license metadata
Arch	✨ New Feature + 🔍 Enhanced	Upstream work is currently .SRCINFO-only, with no .PKGINFO parser, no legacy .AURINFO compatibility, and no Rust implementation	Added .SRCINFO, legacy .AURINFO, and .PKGINFO parsing, real alpm package typing/purls, split-package inheritance, and arch-specific dependency scope preservation	Better Arch/AUR package visibility, proper alpm identities, and fuller source/binary metadata coverage
Ruby	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	External gemspec constants unresolved, Gemfile manifest provenance was dropped, extracted gem duplication, missing nested key-file tagging, and no Ruby-driven license clarity score	Added required-file constant resolution, Gemfile + Gemfile.lock provenance preservation, extracted-gem merge dedupe, nested Ruby file-to-package assignment, key-file tagging, package metadata promotion, and summary license clarity scoring	Better Ruby gemspec parity, stronger Bundler manifest/lockfile provenance, cleaner extracted-gem results, and correct nested key-file clarity
SBT	✨ New Feature + 🔍 Enhanced	No shipped Python SBT parser	Added bounded static build.sbt parsing for top-level literal metadata, literal license/homepage forms, direct/config-prefixed libraryDependencies, root-safe .settings(...), same-file literal Seq(...) bundle reuse, and explicit non-evaluating guardrails	Better JVM/Scala package visibility without executing Scala
Meson	✨ New Feature + 🔍 Enhanced	No shipped Python Meson parser	Added bounded static meson.build parsing for literal project() metadata, top-level dependency() declarations, official license_files/meson_version capture, and explicit no-evaluation guardrails	Better native-project package visibility without executing Meson
Nix	✨ New Feature + 🔍 Enhanced + 🛡️ Security	No shipped Python Nix package parser	Added static flake.nix metadata extraction, flake.lock root-input parsing, bounded default.nix mkDerivation support, and flake sibling assembly without Nix evaluation	Better reproducible-build package visibility with deterministic flake metadata and safe bounded derivation parsing
Helm	✨ New Feature + 🔍 Enhanced	No shipped Python Helm parser	Added static Chart.yaml and Chart.lock parsing, chart metadata extraction, maintainer/dependency preservation, and sibling assembly for declared plus locked Helm dependencies	Better Kubernetes chart package visibility with declared and pinned dependency state
Pixi	✨ New Feature + 🔍 Enhanced	No shipped Python Pixi parser	Added static pixi.toml and version-gated pixi.lock parsing, mixed Conda/PyPI dependency extraction, environment metadata preservation, and sibling assembly for manifest plus lock state	Better reproducible Pixi workspace visibility with direct and locked dependency state
Clojure	✨ New Feature + 🔍 Enhanced	No shipped Python Clojure parser	Added bounded static deps.edn and project.clj parsing, alias/profile dependency extraction, and literal-only guardrails for Leiningen metadata without code evaluation	Better JVM/Clojure package visibility without evaluating project code
Dart	🔍 Enhanced	Scope always null, YAML lossy, lockfile intent flattened, dependency source descriptors dropped, and publish_to / extra pub metadata under-preserved	Proper scope and YAML preservation, lockfile direct/dev/transitive classification, hosted/git/path/sdk descriptor preservation, legacy sdk: compatibility, publish_to: none privacy mapping, archive_url download handling, and richer pubspec metadata capture	Correct dependency classification, better lockfile provenance, and more truthful Dart package metadata
OS Release	🐛 Bug Fix + 🔍 Enhanced	Debian name logic bug + no URL extraction	Fixed name logic + HOME/SUPPORT/BUG URLs	Correct distro identification + richer metadata
Conan Data	🔍 Enhanced	Only extracts primary source URL	Patches metadata + mirror/fallback URLs	Complete source provenance tracking
Docker	✨ New Feature + 🔍 Enhanced	No packagedcode Docker parser for Dockerfile/Containerfile package metadata	Added Dockerfile/Containerfile recognition, OCI label extraction, and non-assembled package data handling	Container metadata visibility and alternative Dockerfile-name support
Deno	✨ New Feature	No Deno parser	Added direct deno.json / deno.jsonc / deno.lock parsing with import-map dependency extraction, v5 lockfile support, and sibling assembly	Better JS/TS dependency visibility for Deno projects
pylock.toml	✨ New Feature	No pylock.toml parser	Added direct PEP 751 / PyPA pylock.toml parsing with dependency graph recovery, provenance extraction, and sibling assembly support	Better Python dependency visibility for standards-based lockfiles
UV Lock	✨ New Feature	No uv.lock parser	Added direct uv lockfile parsing with root-package recovery, dependency groups, resolved packages, and sibling assembly support	Better Python dependency visibility for uv-managed projects
Python	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Rich setup.cfg metadata, unresolved imported-module setup.py dunder patterns, private-package classifiers, installed/source metadata file references, installed wheel provenance, exact-filename pypi.json parsing, pip cache provenance, RFC822 dependency extraction, direct source-distribution archive support, archive-read hardening for suspicious metadata entries, anonymous degraded rows for malformed Python package inputs, and unowned requirements/ subdirectory dependency evidence were incomplete or missing	Added richer setup.cfg metadata, setup.py OrderedDict URL handling, imported-module dunder fallback, private-package detection, installed metadata file-reference collection/assignment, sibling WHEEL enrichment, project-root attachment for requirements/ subdirectory dependency files, direct sdist archive parsing, bound zip validation to actual archive reads, tar/zip archive hardening, datasource-preserving fallback identity for malformed Python package inputs, pip cache origin.json parsing/merge, source SOURCES.txt collection, saved pypi.json parsing, and RFC822 dependency extraction	Better Python manifest fidelity, clearer degraded-input output, safer metadata recovery, richer installed-package provenance, broader PyPI package-input coverage, and stronger archive safety
Swift	🐛 Bug Fix + 🔍 Enhanced	Root-package identity depended too heavily on whichever Swift artifact was available, malformed manifest fallback rows were anonymous, lockfile fallback overstated direct/runtime intent, and local path dependencies were dropped from dumped manifests	Added Swift-specific manifest/show-dependencies/resolved precedence, identified malformed-manifest fallback rows, resolved-only package emission, local path dependency preservation, less-assertive resolved/show-dependencies intent flags, and resolved fallback that enriches manifest-known dependencies instead of flattening every pin into a direct root edge	Correct top-level Swift packages, clearer malformed-manifest output, better dependency fidelity, safer file ownership, and more truthful manifest-vs-lock intent handling
CPAN dist.ini	✨ New Feature + 🔍 Enhanced	Stub-only handler (returns empty)	Full INI parsing with runtime/test/build/configure dependency scopes plus declared-license normalization for Dist::Zilla metadata	Perl Dist::Zilla metadata extraction with correct configure-time dependency semantics and SPDX-aware declared license output
Swift Dependencies	🔍 Enhanced	Only extracts root package name	Full dependency graph with versions + direct/transitive	Complete Swift dependency visibility
Maven	🐛 Bug Fix + ✨ Feature + 🔍 Enhanced	Flattened licenses, missing dependencyManagement/relocation/qualifier support, older POM assumptions, and declared <modules> reactor lists never drove assembly (multi-module source trees stayed per-directory packages only)	Structured licenses + dependencyManagement/relocation/qualifier support + Maven 4.1.0 support + reactor-aware TopologyPlan file ownership that attributes nested module source files (and root-level files) to the correct module package, including nested reactors	Better Maven parity, richer SBOM metadata, modern POM compatibility, and correct file-to-package ownership across multi-module Maven source trees
NuGet	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Missing project-style manifests, bounded CPM property propagation, version backfill for versionless PackageReference dependencies, empty party types, and weak modern nuspec/archive license handling	Added project.json/project.lock.json/PackageReference support, standalone Directory.Packages.props and Directory.Build.props parsing, nearest-ancestor plus bounded parent-import CPM backfill, bounded build-props participation, literal property-backed VersionOverride support, typed parties, modern license hints, and archive-backed license-file extraction	Better .NET package coverage, stronger bounded CPM recovery, and stronger modern NuGet metadata
Cargo	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Missing crate/member file ownership, manifest-declared file references, lockfile dependency provenance, and smaller parser parity gaps around lowercase files plus readme/publish metadata	Added crate/workspace file assignment, explicit Cargo.toml file_references for readme/license-file, preserved Cargo.lock source/checksum provenance, lowercase Cargo filename matching, and readme/publish extraction	Better Rust crate ownership, better manifest/lockfile provenance, better workspace attribution, and tighter Cargo parser parity
Compiled Binary	✨ New Feature + 🔍 Enhanced + 🛡️ Security	Compiled Go and Rust package extraction depends on optional external inspectors in Python core, Python's Windows PE support is a separate narrow handler, and cargo-auditable dependency recovery is not built into the default packagedcode path	Added native core support for Go build-info, Rust cargo-auditable compiled binaries, and bounded Windows PE VERSIONINFO package identity extraction, recovered Rust dependency edges with dedicated binary datasources/PURLs, and bounded oversized Rust audit decompression	Better out-of-the-box compiled-artifact coverage, stronger Rust dependency visibility, narrower Windows executable parity, and safer compiled-binary parsing
ABOUT	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Falls back to invalid pkg:about/... PURLs, misses download_url-based ecosystem inference, and couples file-reference promotion to reopening ABOUT resources	Added real PURL/type inference from download_url, suppressed invalid pkg:about synthesis, preserved ABOUT identity on partial files, and resolved ABOUT file references during scan-time promotion	Better ABOUT package identity, safer fallback behavior, and more reliable referenced-resource promotion
CocoaPods	🐛 Bug Fix + 🔍 Enhanced	Scope semantics remain muddy across pod manifests/lockfiles, and historical duplicate-output regressions need explicit guardrails	Refined scope/runtime semantics across .podspec, .podspec.json, Podfile, and Podfile.lock, and bounded duplicate-output protection for RxDataSources.podspec	More honest dependency semantics and safer protection against explosive duplicate package output
Go	🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced	Missing graph support, thin directive fidelity, and Go test/build file categorization gaps	Added go.mod graph parsing, replace/retract/toolchain directive fidelity, fallback datasource fixes, and build-tag-aware Go source categorization	Better module graph fidelity, safer assembly accounting, and more accurate Go source heuristics
Hackage	✨ New Feature + 🔍 Enhanced	No Haskell package parser for *.cabal, cabal.project, or stack.yaml	Added static parsing for package metadata, component build-depends, project/workspace surfaces, config preservation, and sibling assembly across the three Hackage inputs	Better Haskell package visibility, cleaner project provenance, and assembled multi-surface Hackage results
Hex Lockfile	✨ New Feature	No Hex lockfile parser and no safe lockfile-only Elixir dependency extraction	Added static mix.lock parsing with locked package versions, nested dependency reconstruction, repo/checksum preservation, and explicit no-code-execution scope	Better Elixir/Hex dependency visibility without executable manifest parsing
npm Git URLs	🐛 Bug Fix	Git URLs treated as pinned versions	Correct is_pinned=false for non-version deps	Valid PURLs + correct dependency resolution status
Gitmodules	✨ New Feature	No .gitmodules parser	Full submodule dependency extraction	Complete dependency graphs for projects using submodules
Copyright Detection	🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security	Year range stops at 2039, short-year typo, French/Spanish case bugs, string-based POS tags, global mutable singleton, and more aggressive emitted copyright normalization	Year range 2099, all regex bugs fixed, type-safe enum POS tags, thread-safe LazyLock, shared media metadata clues from supported images and fonts, and source-faithful file-level copyright output by default with an opt-in ScanCode compatibility mode	Correct year detection, reliable i18n, compile-time safety, better compliance fidelity, and parallel scanning
Email/URL Detection	🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security	TLD length too strict, IPv6/private-IP issues, less explicit URL handling	Extended TLD support, robust host/IP filtering, credential stripping, and shared metadata clues from supported images and fonts	Better extraction correctness and safer metadata handling
License Detection	🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security + ⚡ Performance	SPDX-LID fan-out duplicates, stale whole-query cache after SPDX subtraction, bounds-only qcontains/qoverlap, zero-overlap surround merges, per-candidate rule cloning, and global mutable state	SPDX-LID deduplication, immutable whole-query snapshot for AHO, position-set-aware containment/overlap, explicit AHO extra-matchables tracking, unified PositionSpan enum, borrowed rule references in candidates, and thread-safe Arc<LicenseDetectionEngine>	Correct SPDX match counts, clean architectural separation from stale-cache coupling, correct positional overlap semantics, reduced allocation overhead, and safe parallel scanning
CLI Workflows	✨ New Feature + 🐛 Bug Fix	Selected-file and PR-changed-file scans still depend on argv expansion, include-filter approximations, or cwd-sensitive multi-input behavior, and repeated full rescans still pay the full cost on unchanged trees	Added rooted --paths-file native-scan selection with stdin support, root-relative entries, recoverable missing-entry warnings, cwd-independent explicit-root workflows, and opt-in --incremental unchanged-file reuse under a shared cache root	Better PR/changed-file scanning, cleaner CI/container ergonomics, safer selected-file scope control, and faster repeated native rescans
Bazel/Buck build targets	🔍 Enhanced (noise reduction)	Python emits one package per Bazel/Buck BUILD/BUCK target, producing many name-only shells with no license, dependency, or version data	Collapse a build directory's targets into one license-bearing component (sibling-merge), with Bazel consistent with Buck	Bounded, explainable package output; intentionally fewer pkg:bazel/pkg:buck packages than ScanCode, not a regression
Cross-cutting (All Parsers)	🛡️ Security	No DoS limits	File size + iteration limits	Protection against resource exhaustion

Per-Improvement Documentation

Most areas with improvements have a dedicated document. Cross-cutting security hardening and broader infrastructure improvements are documented elsewhere when they span multiple subsystems:

• rpm-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: EVR preservation, hash-based source RPM detection, richer archive metadata, and YumDB enrichment
• debian-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🐛 Bug Fix + ⚡ Performance: direct .deb introspection, embedded copyright recovery, coherent DEP-5 detections, installed Debian sidecar integration, and a Debian-local paragraph parsing optimization
• conan-parser.md — ✨ New Feature: conanfile.txt and conan.lock parsers (Python has neither)
• bitbake-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded .bb + .bbappend parsing, override-style dependency handling, raw declared-license preservation, local file-reference recovery, and sibling assembly support
• carthage-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: Cartfile, Cartfile.private, and Cartfile.resolved parsing with origin-aware dependency metadata and sibling assembly for declared plus pinned state
• cpan-parser.md — ✨ New Feature + 🐛 Bug Fix: Full META.json, META.yml, MANIFEST parsing plus datasource-preserving fallback identity for malformed/unreadable CPAN inputs
• gradle-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: correct Gradle scope classification, all discovered dependency-block parsing, version-catalog alias resolution, Gradle POM license extraction, and nested project-path identifiers
• gradle-module-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: direct Gradle .module parsing with artifact checksums, deduplicated dependency extraction, preserved variant metadata, and file references for published artifacts
• gradle-lockfile-parser.md — ✨ New Feature: gradle.lockfile dependency extraction (Python has no equivalent)
• vcpkg-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: strict-JSON vcpkg.json parsing, project/port manifest support, direct dependency extraction, and embedded/sibling configuration metadata preservation
• bun-lock-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: JSONC-aware bun.lock parsing, static legacy bun.lockb v2 compatibility, and sibling/workspace assembly integration
• arch-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: .SRCINFO, legacy .AURINFO, and .PKGINFO parsing with alpm purls, split-package inheritance, and arch-specific dependency scope preservation
• npm-workspace-parser.md — ✨ New Feature: pnpm-workspace.yaml metadata extraction (Python has stub only)
• npm-yarn-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: platform metadata preservation, scoped URL/metadata normalization fixes, richer Yarn Berry resolution detail, modern lockfile/link handling, Yarn scope inference, and nested/workspace ownership fixes
• npm-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: Yarn resolutions and npm overrides emitted as distinctly scoped pin dependencies (is_pinned = true) with intent booleans left unset per the honest-unknowns guardrail
• structured-metadata-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded CITATION.cff and publiccode.yml parsers for standalone project metadata, SPDX-aware declared-license recovery, and maintainer/author metadata extraction
• composer-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: alternate Composer file names, nested package file assignment, lighter lockfile handling, richer manifest provenance, and declared-license normalization for trustworthy composer.json license metadata
• ruby-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: required-file constant resolution, Gemfile + Gemfile.lock provenance preservation, extracted-gem merge/resource assignment, and nested key-file / license-clarity propagation
• dart-parser.md — 🔍 Enhanced Extraction: proper scope handling, YAML trailing newline preservation, lockfile direct/dev/transitive classification, dependency descriptor preservation, and richer pubspec metadata
• os-release-parser.md — 🐛 Bug Fix + 🔍 Enhanced: Debian name logic fix + URL extraction (HOME, SUPPORT, BUG)
• conan-data-parser.md — 🔍 Enhanced Extraction: Patches metadata + mirror/fallback URL extraction
• docker-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: Dockerfile/Containerfile recognition, OCI label extraction, and non-assembled package data handling
• deno-parser.md — ✨ New Feature: deno.json(c) identity/import parsing, deno.lock v5 extraction, and sibling assembly support
• bazel-module-parser.md — ✨ New Feature: MODULE.bazel parsing with bazel_dep extraction, dev-scope support, and override metadata preservation
• bazel-buck-build-targets.md — 🔍 Enhanced Extraction (noise reduction): collapse Bazel/Buck BUILD/BUCK targets into one component per build directory instead of one empty package per target; intentionally fewer pkg:bazel/pkg:buck packages than ScanCode by design
• go-work-parser.md — ✨ New Feature: go.work parsing with workspace-member recovery, replace metadata extraction, and root-module assembly support
• pylock-toml-parser.md — ✨ New Feature: standardized pylock.toml parsing, dependency graph recovery, provenance extraction, and Python sibling assembly support
• uv-lock-parser.md — ✨ New Feature: uv.lock root-package recovery, dependency groups, resolved package extraction, and Python sibling assembly support
• python-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: richer setup.cfg metadata, setup.py imported-dunder/project-URL fixes, private classifier support, direct sdist archive parsing plus archive hardening, installed/source metadata file-reference handling, sibling WHEEL enrichment, pip cache origin.json provenance, exact-filename pypi.json parsing, and RFC822 dependency extraction
• swift-parser.md — 🐛 Bug Fix + 🔍 Enhanced Extraction: Swift-specific root-package assembly precedence, local path manifest dependency preservation, less-assertive lock/show intent flags, resolved fallback enrichment, and nested-root-safe file ownership
• cpan-dist-ini-parser.md — ✨ New Feature + 🔍 Enhanced: Full dist.ini parsing with runtime/test/build/configure dependency extraction (Python has stub only)
• swift-show-dependencies-parser.md — 🔍 Enhanced Extraction: Full dependency graph with versions, direct/transitive marking, and less-assertive runtime intent
• rpm-specfile-parser.md — ✨ New Feature: Full RPM spec preamble parsing (Python is stub with TODO)
• cpan-makefile-pl-parser.md — ✨ New Feature + 🔍 Enhanced: Makefile.PL WriteMakefile extraction plus bounded VERSION_FROM / ABSTRACT_FROM recovery (Python has no parse method)
• osgi-manifest-parser.md — ✨ New Feature: OSGi bundle metadata extraction with mixed-manifest Maven identity fallback when strong Maven coordinates are present (Python has empty patterns)
• maven-parser.md - 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: structured licenses, dependencyManagement/relocation/qualifier support, Maven 4.1.0 support, richer normalized metadata, and reactor/multi-module TopologyPlan file-ownership assignment
• nuget-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: nearest-ancestor NuGet CPM backfill plus bounded parent-import/build-props participation, literal VersionOverride support for PackageReference dependencies in project files, standalone Directory.Packages.props / Directory.Build.props parsing, modern nuspec license hints, .nupkg license-file extraction, .deps.json runtime graph parsing, and support for legacy/project-style NuGet manifests
• cargo-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: crate/workspace file ownership, Cargo.toml file references, Cargo.lock provenance preservation, lowercase Cargo file matching, and readme/publish manifest parity
• compiled-binary-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🛡️ Security: native Go build-info, Rust cargo-auditable, and bounded Windows PE version-resource extraction in core scanning, Rust dependency-edge recovery, and bounded Rust audit decompression
• go-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: go.mod graph support, replace/retract/toolchain directive fidelity, and Go test/build source categorization
• hackage-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static *.cabal / cabal.project / stack.yaml parsing with component dependency recovery, project-config preservation, and sibling assembly support
• sbt-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static build.sbt parsing for top-level literal metadata, literal license/homepage forms, literal libraryDependencies, limited string alias reuse, and explicit no-evaluation guardrails
• meson-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static meson.build parsing for literal project() metadata, top-level dependency() declarations, official license_files/meson_version capture, and explicit no-evaluation guardrails
• nix-parser.md — ✨ New Feature + 🔍 Enhanced Extraction + 🛡️ Security: static flake.nix / flake.lock support, bounded default.nix mkDerivation extraction, and flake sibling assembly without Nix evaluation
• helm-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static Chart.yaml / Chart.lock parsing with chart metadata recovery, declared dependency extraction, pinned lock metadata, and sibling assembly support
• pixi-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: static pixi.toml / version-gated pixi.lock parsing with workspace metadata recovery, mixed Conda/PyPI dependency extraction, environment metadata preservation, and sibling assembly support
• clojure-parser.md — ✨ New Feature + 🔍 Enhanced Extraction: bounded static deps.edn / project.clj parsing with Maven/Clojars-style dependency extraction, alias/profile support, and explicit no-evaluation guardrails
• hex-lock-parser.md — ✨ New Feature: static mix.lock parsing with locked Hex dependency extraction and nested dependency reconstruction under a no-code-execution scope
• conda-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: conda-meta file assignment and channel-vs-channel_url disambiguation
• alpine-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: APKBUILD recipe parsing, raw matched-text preservation for custom:multiple, retained fileless packages, and HTTPS Alpine VCS URLs
• about-parser.md — 🐛 Bug Fix + ✨ New Feature + 🔍 Enhanced Extraction: PURL/type inference from download_url, invalid pkg:about suppression, graceful partial ABOUT handling, and scan-time file-reference resolution
• cocoapods-parser.md — 🐛 Bug Fix + 🔍 Enhanced Extraction: refined dependency scope semantics and bounded duplicate-output protection for RxDataSources.podspec
• npm-git-url-dependencies.md — 🐛 Bug Fix: Correct handling of Git URLs, GitHub shortcuts, and local paths (Python treats them as pinned versions)
• gitmodules-parser.md — ✨ New Feature: Git submodule dependency extraction (Python has no equivalent parser)
• copyright-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security: Year range fix, regex typo fixes, type-safe POS tags, thread-safe design, and source-faithful copyright output by default
• email-url-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security: Email/URL extraction hardening, stronger filtering, and EXIF/XMP-backed metadata detection
• license-detection.md — 🐛 Bug Fix + 🔍 Enhanced + 🛡️ Security + ⚡ Performance: SPDX-LID deduplication, position-set-aware containment/overlap, immutable whole-query snapshot for AHO, explicit extra-matchables tracking, unified PositionSpan, and borrowed rule references in candidates
• cli-workflows.md — ✨ New Feature + 🐛 Bug Fix: rooted --paths-file selected-file scanning plus opt-in --incremental rescans with cache-root controls for repeated native workflows

Related broader architecture/security docs:

• Cross-cutting parser security hardening - File size limits, iteration limits, and archive-safety policy are documented in ADR 0004: Security-First Parsing.
• Incremental rescans - The implemented unchanged-file reuse workflow is documented in ARCHITECTURE.md.
• Scan-result shaping parity - The shaping pipeline and landed behavior are documented in ARCHITECTURE.md#post-processing-pipeline.

Contributing Improvements

For the general contributor workflow, local setup, testing guidance, and pull request conventions, start with ../CONTRIBUTING.md.

When implementing a parser or detection subsystem, if you discover:

1. A bug in Python: Fix it in Rust, document here
2. A TODO in Python: Implement it in Rust, document here
3. Missing extraction: Add it in Rust, document here

Template for new improvement docs: See existing files for structure.

Related Documentation

• ADR 0002: Extraction vs Detection Separation - Why we separate concerns better than Python
• ADR 0004: Security-First Parsing - Our security advantages over Python
• SUPPORTED_FORMATS.md - Auto-generated list of all supported formats