gotextfsm

This is a Golang implementation (or version) of Google's TextFSM Python library.

TextFSM is a template-based state machine for parsing semi-formatted text. It was originally developed to allow programmatic access to information returned from the command line interface (CLI) of networking devices.

The complete documentation of TextFSM can be found at the TextFSM Wiki.

This porting attempts to be 100% compatible to the original TextFSM specification given in the above links.

Get Started

Get the library:

$ go get github.com/sirikothe/gotextfsm

Basic Usage

Import the gotextfsm library:

import "github.com/sirikothe/gotextfsm"

Create a TextFSM Object and parse the template:

  fsm := gotextfsm.TextFSM{}
  # template should hold the string of the Textfsm to be parsed.
  err := fsm.ParseString(template)
  # err will be nil if the parsing of the template is successful. 
  # If will contain error object if the parsing failed.

Parse a raw input string using the fsm object created:

  parser := gotextfsm.ParserOutput{}
  err = parser.ParseTextString(input, fsm, true)
  # err will be nil if the parsing of the input (as per the fsm provided) is successful. 
  # If will contain error object if the parsing failed.

At the end of the parsing of input the parser's Dict object contains the results.

Complete Example Code

package main

import (
	"fmt"

	"github.com/sirikothe/gotextfsm"
)

func main() {
	template := `Value beer (.*)

Start
  ^$beer
`
	input := "Hello World"
	fsm := gotextfsm.TextFSM{}
	err := fsm.ParseString(template)
	if err != nil {
		fmt.Printf("Error while parsing template '%s'\n", err.Error())
		return
	}
	parser := gotextfsm.ParserOutput{}
	err = parser.ParseTextString(input, fsm, true)
	if err != nil {
		fmt.Printf("Error while parsing input '%s'\n", err.Error())
	}
	fmt.Printf("Parsed output: %v\n", parser.Dict)
}

How to read results of parsing

The defined type for ParserOutput.Dict is []map[string]interface{}.

This is an array of maps. Each element in array represents a record of parsed output.

Each record is of type map[string]interface{} where the key is the field name (type string) and value is the field value (type interface{}).

Even though the field value is defined as type interface{}, its concrete type is one of:

• []string type -> For variables declared as List in the definition. ex. Value List ifnames (\w+)
• map[string]string type -> For variables declared as scalar, but with nested regexes. ex. Value person ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
• []map[string]string type -> For variables declared as List, but with nested regexes. ex. Value List person ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
• string type -> For every other variable type. This is most common use case.

Option 1 - Example code to handle the output

Following complete code snippet shows an example of how to process the output of parser.

package main

import (
	"fmt"

	"github.com/sirikothe/gotextfsm"
)

func main() {
	template := `Value continent (.*)
Value List countries (.*)
Value state_abbr ((?P<fullstate>\w+):\s+(?P<abbr>\w{2}))
Value List persons ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)

Start
	^Continent: ${continent}
	^Country: ${countries}
	^State: ${state_abbr}
	^${persons}
`

	input := `Continent: North America
Country: USA
Country: Canada
Country: Mexico
State: California: CA
Siri: 50 CA
Raj: 22 NM
Gandhi: 150 NV
`

	fsm := gotextfsm.TextFSM{}
	err := fsm.ParseString(template)
	if err != nil {
		fmt.Printf("Error while parsing template '%s'\n", err.Error())
		return
	}
	parser := gotextfsm.ParserOutput{}
	err = parser.ParseTextString(input, fsm, true)
	if err != nil {
		fmt.Printf("Error while parsing input '%s'\n", err.Error())
	}
	for _, record := range parser.Dict {
		for key, value := range record {
			switch value.(type) {
			case string:
				// typecast to string and use it.
				// ex: Value continent (.*)
				fmt.Printf("%s: %s\n", key, value.(string))
			case []string:
				// List type variable. typecast to []string and use it.
				// ex: Value List countries (.*)
				fmt.Printf("%s: %s\n", key, value.([]string))

			case map[string]string:
				// Nested scalar variable.
				// ex: Value state_abbr ((?P<fullstate>\w+):\s+(?P<abbr>\w{2}))
				fmt.Printf("%s: %s\n", key, value.(map[string]string))

			case []map[string]string:
				// Nested List variable.
				// ex: Value List persons ((?P<name>\w+):\s+(?P<age>\d+)\s+(?P<state>\w{2})\s*)
				fmt.Printf("%s: %s\n", key, value.([]map[string]string))
			default:
				// Should never happen.
				panic("Really?")
			}
		}
	}
}

Option 2 - Example code to handle the output

You can also marshal the resulting dict to json (or yaml) if that makes it easier for you to handle the output.

	str, err := json.Marshal(parser.Dict)
	if err != nil {
		fmt.Printf("Unable to convert dict to json \n", err)
	} else {
		fmt.Printf("JSON: %s\n", str)
	}

Output from the above example:

JSON: [{"continent":"North America","countries":["USA","Canada","Mexico"],"persons":[{"age":"50","name":"Siri","state":"CA"},{"age":"22","name":"Raj","state":"NM"},{"age":"150","name":"Gandhi","state":"NV"}],"state_abbr":{"abbr":"CA","fullstate":"California"}}]

Highlights

• Attempts to be 100% compatible with the original TextFSM implementation (See differences section).
• Very nimble code with zero external dependencies on any other libraries.
• Well tested (~ 97% code coverage) >1740 Test cases executed!!!
  • All test cases of Python's implementation are ported and executed.
  • More test cases added as well to test corner cases.
• All the test cases of ntc-templates are executed.
  • Out of 1578 test cases of ntc-templates, 28 of them are failing (All due to reasons listed in Caveats).

Differences with Python's implementation

The following are the differences between this implementation of TextFSM and the original Python implementation:

• Python's implementation provides 2 ways of getting results.
  • Output as a list of lists.
    • The outer list represents a record and inner list contains the values in the order they were declared.
  • Output as a list of dicts.
    • This Golang implementation provides the output as only slice of maps. It does not provide a slice of slices.
• [TODO] :construction: This Golang implementation (currently) implements the core TextFSM functionality. It does not implement the following:
  • clitable
  • terminal
  • texttable

Caveats

There are differences in Golang's regular expression implementation and Python's.

:warning: Because of this reason some templates that are valid in Python fail parsing in gotextfsm.

Testing

PS C:\Users\siri\code\nuviso\GitHub\gotextfsm> go test -v
=== RUN   TestParseText
    parsetext_test.go:83: Executed 48 test cases
--- PASS: TestParseText (0.04s)
=== RUN   TestPyTemplate
    pytemplate_test.go:31: Executed 9 test cases
--- PASS: TestPyTemplate (0.00s)
=== RUN   TestRuleParse
    rule_test.go:41: Executed 25 test cases
--- PASS: TestRuleParse (0.00s)
    textfsm_test.go:84: Executed 56 test cases
--- PASS: TestFSMParse (0.01s)
=== RUN   TestValueParse
    value_test.go:56: Executed 27 test cases
--- PASS: TestValueParse (0.01s)
PASS
ok      _/C_/Users/siri/code/nuviso/GitHub/gotextfsm    0.235s
PS C:\Users\siri\code\nuviso\GitHub\gotextfsm> go test -cover
PASS
coverage: 96.9% of statements
ok      _/C_/Users/siri/code/nuviso/GitHub/gotextfsm    0.236s