text-processing.md

July 15, 2021 · View on GitHub

Bookmarks tagged [text-processing]

www.codever.land/bookmarks/t/text-processing

TankerHQ/ruplacer

https://github.com/TankerHQ/ruplacer

Find and replace text in source files

lavifb/todo_r

https://github.com/lavifb/todo_r

Find all your TODO notes with one command!

whitfin/runiq

https://github.com/whitfin/runiq

an efficient way to filter duplicate lines from unsorted input.

whitfin/bytelines

https://github.com/whitfin/bytelines

Read input lines as byte slices for high efficiency.

vishaltelangre/ff

https://github.com/vishaltelangre/ff

Find files (ff) by name!

BurntSushi/suffix

https://github.com/BurntSushi/suffix

Linear time suffix array construction (with Unicode support)

BurntSushi/tabwriter

https://github.com/BurntSushi/tabwriter

Elastic tab stops (i.e., text column alignment)

pwoolcoc/ngrams

https://github.com/pwoolcoc/ngrams

Construct n-grams from arbitrary iterators

ps1dr3x/easy_reader

https://github.com/ps1dr3x/easy_reader

A reader that allows forwards, backwards and random navigations through the lines of huge files without consuming iterators [...

rust-lang/regex

https://github.com/rust-lang/regex

Regular expressions (RE2 style)

greyblake/whatlang-rs

https://github.com/greyblake/whatlang-rs

Natural language detection library based on trigrams

yaa110/rake-rs

https://github.com/yaa110/rake-rs

Multilingual implementation of RAKE algorithm for Rust

align

https://github.com/Guitarbum722/align

A general purpose application that aligns text.

allot

https://github.com/sbstjn/allot

Placeholder and wildcard text parsing for CLI tools and bots.

bbConvert

https://github.com/CalebQ42/bbConvert

Converts bbCode to HTML that allows you to add support for custom bbCode tags.

blackfriday

https://github.com/russross/blackfriday

Markdown processor in Go.

bluemonday

https://github.com/microcosm-cc/bluemonday

HTML Sanitizer.

codetree

https://github.com/aerogo/codetree

Parses indented code (python, pixy, scarlet, etc.) and returns a tree structure.

colly

https://github.com/asciimoo/colly

Fast and Elegant Scraping Framework for Gophers.

commonregex

https://github.com/mingrammer/commonregex

A collection of common regular expressions for Go.

dataflowkit

https://github.com/slotix/dataflowkit

Web scraping Framework to turn websites into structured data.

did

https://github.com/ockam-network/did

DID (Decentralized Identifiers) Parser and Stringer in Go.

doi

https://github.com/hscells/doi

Document object identifier (doi) parser in Go.

editorconfig-core-go

https://github.com/editorconfig/editorconfig-core-go

Editorconfig file parser and manipulator for Go.

enca

https://github.com/endeveit/enca

Minimal cgo bindings for libenca.

encdec

https://github.com/mickep76/encdec

Package provides a generic interface to encoders and decodersa.

genex

https://github.com/alixaxel/genex

Count and expand Regular Expressions into all matching Strings.

github_flavored_markdown

https://godoc.org/github.com/shurcooL/github_flavored_markdown

GitHub Flavored Markdown renderer (using blackfriday) with fenced code block highlighting, clickable header anchor links.

go-fixedwidth

https://github.com/ianlopshire/go-fixedwidth

Fixed-width text formatting (encoder/decoder with reflection).

go-humanize

https://github.com/dustin/go-humanize

Formatters for time, numbers, and memory size to human readable format.

go-nmea

https://github.com/adrianmo/go-nmea

NMEA parser library for the Go language.

go-runewidth

https://github.com/mattn/go-runewidth

Functions to get fixed width of the character or string.

go-slugify

https://github.com/mozillazg/go-slugify

Make pretty slug with multiple languages support.

go-toml

https://github.com/pelletier/go-toml

Go library for the TOML format with query support and handy cli tools.

go-vcard

https://github.com/emersion/go-vcard

Parse and format vCard.

go-zero-width

https://github.com/trubitsyn/go-zero-width

Zero-width character detection and removal for Go.

gofeed

https://github.com/mmcdole/gofeed

Parse RSS and Atom feeds in Go.

gographviz

https://github.com/awalterschulze/gographviz

Parses the Graphviz DOT language.

gommon/bytes

https://github.com/labstack/gommon/tree/master/bytes

Format bytes to string.

gonameparts

https://github.com/polera/gonameparts

Parses human names into individual name parts.

goq

https://github.com/andrewstuart/goq

Declarative unmarshaling of HTML using struct tags with jQuery syntax (uses GoQuery).

GoQuery

https://github.com/PuerkitoBio/goquery

GoQuery brings a syntax and a set of features similar to jQuery to the Go language.

goregen

https://github.com/zach-klippenstein/goregen

Library for generating random strings from regular expressions.

gotext

https://github.com/leonelquinteros/gotext

GNU gettext utilities for Go.

guesslanguage

https://github.com/endeveit/guesslanguage

Functions to determine the natural language of a unicode text.

htmlquery

https://github.com/antchfx/htmlquery

An XPath query package for HTML, lets you extract data or evaluate from HTML documents by an XPath expression.

inject

https://github.com/facebookgo/inject

Package inject provides a reflect based injector.

ltsv

https://github.com/Wing924/ltsv

High performance LTSV (Labeled Tab Separeted Value) reader for Go.

mxj

https://github.com/clbanning/mxj

Encode / decode XML as JSON or map[string]interface{}; extract values with dot-notation paths and wildcards. Replaces x2j and j2x packages.

sdp

https://github.com/gortc/sdp

SDP: Session Description Protocol [RFC 4566].

sh

https://github.com/mvdan/sh

Shell parser and formatter.

slug

https://github.com/gosimple/slug

URL-friendly slugify with multiple languages support.

Slugify

https://github.com/avelino/slugify

Go slugify application that handles string.

syndfeed

https://github.com/zhengchun/syndfeed

A syndication feed for Atom 1.0 and RSS 2.0.

toml

https://github.com/BurntSushi/toml

TOML configuration format (encoder/decoder with reflection).

gofuckyourself

https://github.com/JoshuaDoes/gofuckyourself

A sanitization-based swear filter for Go.

gotabulate

https://github.com/bndr/gotabulate

Easily pretty-print your tabular data with Go.

kace

https://github.com/codemodus/kace

Common case conversions covering common initialisms.

parseargs-go

https://github.com/nproc/parseargs-go

string argument parser that understands quotes and backslashes.

parth

https://github.com/codemodus/parth

URL path segmentation parsing.

radix

https://github.com/yourbasic/radix

fast string sorting algorithm.

TySug

https://github.com/Dynom/TySug

Alternative suggestions with respect to keyboard layouts.

xj2go

https://github.com/stackerzzq/xj2go

Convert xml or json to go struct.

xurls

https://github.com/mvdan/xurls

Extract urls from text.

chardet

https://github.com/chardet/chardet

Python 2/3 compatible character encoding detector.

difflib

https://docs.python.org/3/library/difflib.html

(Python standard library) Helpers for computing deltas.

ftfy

https://github.com/LuminosoInsight/python-ftfy

Makes Unicode text less broken and more consistent automagically.

fuzzywuzzy

https://github.com/seatgeek/fuzzywuzzy

Fuzzy String Matching.

Levenshtein

https://github.com/ztane/python-Levenshtein/

Fast computation of Levenshtein distance and string similarity.

pangu.py

https://github.com/vinta/pangu.py

Paranoid text spacing.

pyfiglet

https://github.com/pwaller/pyfiglet

An implementation of figlet written in Python.

pypinyin

https://github.com/mozillazg/python-pinyin

Convert Chinese hanzi (漢字) to pinyin (拼音).

textdistance

https://github.com/orsinium/textdistance

Compute distance between sequences. 30+ algorithms, pure python implementation, common interface, optional external libs usage.

unidecode

https://pypi.python.org/pypi/Unidecode

ASCII transliterations of Unicode text.

awesome-slugify

https://github.com/dimka665/awesome-slugify

A Python slugify library that can preserve unicode.

python-slugify

https://github.com/un33k/python-slugify

A Python slugify library that translates unicode to ASCII.

unicode-slugify

https://github.com/mozilla/unicode-slugify

A slugifier that generates unicode slugs with Django as a dependency.

hashids

https://github.com/davidaurelio/hashids-python

Implementation of hashids in Python.

shortuuid

https://github.com/skorokithakis/shortuuid

A generator library for concise, unambiguous and URL-safe UUIDs.

ply

https://github.com/dabeaz/ply

Implementation of lex and yacc parsing tools for Python.

pyparsing

https://github.com/pyparsing/pyparsing

A general purpose framework for generating parsers.

python-nameparser

https://github.com/derek73/python-nameparser

Parsing human names into their individual components.

python-phonenumbers

https://github.com/daviddrysdale/python-phonenumbers

Parsing, formatting, storing and validating international phone numbers.

python-user-agents

https://github.com/selwin/python-user-agents

Browser user agent parser.

sqlparse

https://github.com/andialbrecht/sqlparse

A non-validating SQL parser.