Kilotest as connector to and collection of AI Tools

September 15, 2026 · View on GitHub

Introduction

Until 2026 Kilotest was intended, and implemented, as a web application performing a service for human users.

Beginning in May 2026, it became evident that Kilotest could operate as a set of tools for language models to allow them to improve their answers to questions about the front-end quality (accessibility, usability, and standards conformity) of specific web pages.

A decision was made to make Kilotest discoverable, usable, and, where appropriate, used as a provider of tools for language models.

Terms

  • Language model: A model (e.g., Claude Haiku 4.5, Kimi K2.7, GPT-5.4, Gemini 3.1 Pro) that can consume and generate text, images, or other content.
  • AI platform: A platform (e.g., Claude Desktop, Perplexity, ChatGPT, Gemini 3.5 Flash) that gives human users access to the services of language models and connects language models to productivity resources.
  • tool: A specialized productivity resource providing a capability that a language model needs but does not have. In the OpenAPI specification, the term operation is used to mean tool.
  • connector: A service that allows AI platforms to enable their language models to discover, evaluate, and use tools.

Internal features

The internal features that make Kilotest a collection of tools for language models are:

  • An API, with specific functional and utility modules in the api directory.
  • Tests of the functionalities of the API.
  • A JSON-LD script in the index.html file, providing structured data about the Kilotest API.
  • Environment variables in the env.example file.
  • An llms.txt file and an llms-full.txt file, documenting the use of Kilotest by language models, conforming to the llms-txt specification.
  • A generated openapi.yaml file, documenting the Kilotest API, conforming to the OpenAPI specification.
  • An mcp.js file, providing an MCP server for Kilotest.
  • A server.json file, conforming to the MCP server schema, and a GitHub Actions workflow (publish-mcp.yml) that authenticates via GitHub OIDC and publishes that file to the official MCP Registry whenever server.json or api/version.ts changes on the main branch.
  • A capability.md file with a URL, summarizing what Kilotest tools can do.
  • mcp, mcp-server, and modelcontextprotocol keywords in package.json, supporting discovery of the @jrpool/kilotest npm package by tools and humans searching the npm registry for MCP servers.
  • An lhm.plugin.json file, conforming to the LobeHub plugin manifest format, declaring Kilotest as a plugin in the LobeHub MCP Market. Publishing and updating are non-automated (lhm plugin publish/update), because LobeHub requires browser-based OAuth and does not support machine-to-machine publishing.
  • A sitemap.xml file.
  • Documentation in the README.md file.
  • This AI-TOOLS.md file.
  • A tutorial on authorizing language models to use Kilotest.

External features

The external features that support the use of Kilotest as a collection of AI tools are:

  • A pull request to add Kilotest to the list of public APIs in the public-apis repository.

  • An issue to add Kilotest to openapi-directory.

  • A pull request to add Kilotest to the WAI evaluation tools list.

  • Registration of Kilotest as an active server in the official MCP Registry (io.github.jrpool/kilotest), maintained by the Model Context Protocol project. The PulseMCP directory and the VS Code MCP gallery both crawl this registry, so Kilotest is discoverable through them with no separate submission.

  • Registration of Kilotest with the Smithery MCP server registry.

  • Registration of Kilotest with the Glama MCP server registry, including a claim of ownership of the MCP connector by the maintainer via a TXT DNS record for _glama-claim.kilotest.com, with the value glama_claim_gVV4HrPdcXr9TEfj1p3icEUBJ3_OVJGX.

  • Registration of Kilotest with Awesome MCP Servers.

  • Registration of Kilotest with the RapidAPI Hub.

  • Registration of Kilotest with the LobeHub MCP Market (identifier: jrpool-kilotest), enabled with an lhm.plugin.json file.

  • Deployment of an MCP server in HTTP mode on the Kilotest service host.

  • Configuration of Claude Desktop on the local development host and the claude.ai web application to connect Claude Desktop models to the Kilotest MCP server. The configuration was performed in the UI of each platform with the addition of Kilotest as a connector. The user used the Customize/Connectors/Add connector/Add custom connector interface, providing these data before activating the Add button:

    • Name: Kilotest
    • Remote MCP server URL: https://kilotest.com/mcp

  • MCP-related topics added to the repository details.

  • An MCP Registry badge added to the README.md file.

  • A link to the MCP server on the README.md page of the testaro-issues repository.

  • Registration of Kilotest and its sitemap at Google Search Console and Bing Webmaster Tools.

Required maintenance

Whenever the /api/version.ts file is updated:

  • Update the server.json file (version field).
  • Update the lhm.plugin.json file (version field).
  • Regenerate openapi.yaml (npm run generate:openapi).
  • Create a pull request and merge to main (triggers publish-mcp.yml automatically for the MCP Registry).
  • Execute lhm plugin update --dir . (for LobeHub).

Use cases

The rationale for Kilotest as a collection of tools for language models is set forth in the llms-full.txt file and is not repeated here.

Some common anticipated use cases for this role are:

  1. A user of a web-builder platform with responsibility for a website asks an AI platform for help in creating or improving the website.
  2. A prospective customer of a web development service asks an AI platform to evaluate the quality of websites in the portfolios of candidate vendors.
  3. A professional web developer within an organization asks an AI platform for a code review.
  4. A risk-management professional within an organization asks an AI platform to report on any web accessibility defects that could expose the organization to prosecution or civil litigation for disability discrimination.
  5. A person who depends on web accessibility because of disabilities asks an AI platform to provide documentary support for a complaint to the owner of a website about accessibility defects.
  6. A disability-rights advocate or attorney concerned with inaccessibility in a particular industry asks an AI platform to perform a front-end-quality comparison of some websites in that industry.

Among these use cases, case 3 would make it feasible for the user to tell an AI platform explicitly and formally that Kilotest is an available connector to tools that are relevant to the task. All 5 of the other use cases would not make that feasible. In those 5 use cases, the user has a question but relies on the AI platform to know or discover which relevant connectors exist, to select appropriate connectors, and to provide language models that can use those connectors and tools that they connect to.

Use cases 1, 2, 4, 5, and 6 exemplify a widespread expectation and demand for AI platform capability. The commonality is: “I have a question; answer it.” If Kilotest can be employed as an expert for AI platforms in relevant cases, platforms will be more successful in satisfying that demand. At present this is a difficult problem because of platform limitations and a lack of standardization.

Future work

To-dos recommended by Claude Sonnet 5 Medium on Devin:

  • No content/backlink outreach. Registrations are passive listings; there's no blog post, Show HN, or subreddit post (r/modelcontextprotocol, r/accessibility, r/webdev) building external backlinks and social-discovery signal, which the doc's own "difficult problem…lack of standardization" framing suggests is exactly the gap left for the human-driven parts of discoverability.

  • Other directories not yet targeted. The mcp.so directory offers a free tier but steers submitters toward a $39 paid listing; deferred as a commercial decision. The Docker MCP Catalog is not a fit because it requires stdio transport and a Dockerfile, and Kilotest is streamable-http.

  • Client-onboarding friction isn't documented. Some MCP clients need the mcp-remote npx bridge to use a remote streamable-HTTP server like Kilotest's. Ready-made Claude Desktop/Cursor/Windsurf config instructions added to QAI would convert "discovered" into "used" faster, since case 3 of your use cases assumes a technically savvy discoverer but not all users are.

Other future work:

  • Add to QAI, or replace much of the current QAI content with, an Add Custom Connector link.

  • Consider subscribing to a Claude.ai Team plan and submitting Kilotest to the Claude Connectors Directory. Details provided by GLM-5.2 High: “If accepted, Kilotest would get a permanent listing URL like https://claude.ai/directory/connectors/kilotest with a Connect button, and would appear in Claude’s in-product browse/search and ‘Suggested Connectors’. This is the highest-value listing for your use cases, because it puts Kilotest directly in front of Claude users who have not explicitly searched for it.”

  • Recommended by GLM-5.2 High: Submit a Kilotest-appropriate request to one or more AI platforms and discover whether the model discovers Kilotest and, if so, either uses it or offers to use it in the creation of a response.

  • Recommended by ChatGPT: A general capability manifest that says, in one machine-readable place: Here are the capabilities Kilotest provides; here are all the interfaces through which you can invoke them; here are their requirements and effects. For example:

    Capability: web-page-quality-audit

Purpose:
  Test a public web page for accessibility,
  usability, HTML and web-standards problems.

Interfaces:
  MCP: https://kilotest.com/mcp
  OpenAPI: https://kilotest.com/openapi.yaml
  Web UI: https://kilotest.com/

Input:
  Public URL

Output:
  Structured findings, affected elements,
  rule-engine diagnoses, WCAG mappings

Side effects:
  Reads the specified public web page.
  Does not modify the target site.

Cost:
  ...