TypeAgent Shell

Overview

TypeAgent Shell is an UI entry point to TypeAgent sample code that explores architectures for building interactive agents with natural language interfaces using structured prompting and LLM.

TypeAgent Shell is a personal agent that takes user request and use an extensible set of agents to perform actions, answer questions, and carry a conversation. TypeAgent CLI's interactive mode is a command line version of the app, and both shared the core dispatcher component. Please read dispatcher's README.md on example requests and usage.

Prerequisites

Linux/WSL Build Prerequisites

TypeAgent shell is built using Electron. Follow these instructions to install the libraries needed to build and use in the Linux/WSL environment.

Running the agent shell

pnpm run shell

On Windows, if you notice a lag when starting up the shell, you can add the source code folder to the exclusions list for Windows Defender by following these instructions.

Additionally, if you are running MS Graph based sample agents like Calendar and Email, there is an auth token persisted in the identity cache that can occasionally get corrupted. This could also slow down the start up time of the shell, you can delete that by running:

del %LOCALAPPDATA%\.IdentityService\typeagent-tokencache

Conversation Management

When connected to the agent server (remote mode), TypeAgent Shell supports full multi-conversation management. Conversations persist across sessions and can be created, renamed, switched, and deleted from the UI.

Default conversation

When the Shell connects to the agent server, it automatically joins a conversation named "Shell", creating it if it does not already exist. This conversation persists across restarts — history is replayed on reconnect, with past messages shown in grayscale and a ─── now ─── separator marking where new activity begins.

Conversation commands

Use /conversation (or the @conversation alias) to manage conversations from the chat input. All three input styles route through the same dispatcher action, so you can also drive these via natural language (e.g., "list my conversations", "create a new conversation called notes"):

Command	Description
/conversation	Show the help screen (same as /conversation help)
/conversation help	Show the list of conversation commands
/conversation list	List all conversations; the active one is marked with ← current
/conversation info	Show the active conversation's name and id
/conversation new [name]	Create a new conversation and switch to it (auto-names when no name is given)
/conversation switch <id|name>	Switch to a conversation by id or name (case-insensitive)
/conversation next	Switch to the next conversation in the server's list (wraps around)
/conversation prev	Switch to the previous conversation in the server's list (wraps around)
/conversation rename [id|name] <newName>	Rename a conversation (omit [id|name] to rename the active conversation)
/conversation delete <id|name>	Delete a conversation by id or name (the server rejects deleting the active one)

Switching conversations

When you switch to a different conversation:

• The chat area clears and replays that conversation's history, shown in grayscale to distinguish it from new activity.
• A ─── now ─── separator marks where new messages will appear.
• A status message confirms which conversation you have joined: Connected to conversation 'X'.

Request queue affordances

When you submit a request while one is already running, it is queued on the server and surfaced inline on the corresponding user bubble:

• A small "queued" chip appears on each waiting bubble. Click the × on the chip to cancel that specific queued entry without affecting the running one.
• The bubble for the currently-executing request shows a "running" chip.
• Esc with focus in the input cancels the in-flight request (chat-ui's built-in stop affordance — equivalent to clicking the stop button on the input).
• Esc twice within 1 second (from anywhere in the window) cancels every queued + running entry on the current conversation.

These chips also appear for entries submitted by other clients connected to the same conversation, so you can see and cancel work driven by a peer Shell or CLI.

Multi-client notifications

If you have multiple clients connected (e.g., both a Shell and a CLI connected to the same conversation), you will see status messages when clients join or leave:

[A new client has joined this conversation. You are connected to 'my-chat'.]
[A client has left this conversation. You remain connected to 'my-chat'.]

Local mode

In local mode (no agent server), only a single default conversation is available. Conversation switching and creation are not supported.

In local mode, the shell also hosts an in-process port-discovery WebSocket on ws://localhost:8999/ (the same default the standalone agentServer uses) so external clients like the Chrome extension can find in-process agents — e.g. the browser agent's dynamically assigned WebSocket port — through the same discovery channel they would use against a real agentServer. If port 8999 is already in use (a real agentServer running, another local-mode shell, etc.) the bind fails loudly and the shell continues without discovery; external clients won't be able to find in-process agents until you restart the shell with no conflict.

Azure Speech to Text service (Optional)

Currently, TypeAgent Shell optionally supports voice input via Azure Speech Services or Local Whisper Service in addition to keyboard input.

To set up Azure Speech to Text service, the following variables in config.local.yaml (under speech) or the legacy .env are needed:

Variable	Value
SPEECH_SDK_ENDPOINT	Service URL (or speech API resource ID when using Identity based authentication)
SPEECH_SDK_KEY	API key
SPEECH_SDK_REGION	Region of the service (e.g. westus2)

Keyless API Access

If you would like to enable keyless Speech API access you must have performed the following steps:

1. Specify identity as the SPEECH_SDK_KEY in config.local.yaml (set speech.key: identity) or the legacy .env file.
2. Replace the SPEECH_SDK_ENDPOINT value with the azure resource id of your cognitive service instance (i.e. /subscriptions/<your subscription guid>/resourceGroups/myResourceGroup/providers/Microsoft.CognitiveServices/accounts/speechapi).
3. Configure your speech API to support Azure Entra RBAC and add the necessary users/groups with the necessary permissions (typically Cognitive Services Speech User or Cognitive Services Speech Contributor). More information on cognitive services roles here.
4. If you are using JIT access elevate prior to calling the speech API. Please refer to the elevate.js script for doing this efficiently.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.