Tool Reliability Improvements Summary

June 27, 2025 · View on GitHub

This document summarizes the reliability and fault tolerance improvements made to the OCode tools.

Completed Improvements (6/10 Tasks)

1. ✅ Centralized Path Validation Utility

File: ocode_python/utils/path_validator.py

  • Prevents path traversal attacks with comprehensive validation
  • Handles platform-specific concerns (Windows reserved names, dangerous characters)
  • Validates against base paths to prevent directory escapes
  • Checks for null bytes, symlinks, and dangerous patterns
  • Integrated into: FileReadTool, FileWriteTool, and other file operations

2. ✅ Proper Async Context Managers for BashTool

File: ocode_python/tools/bash_tool.py

  • Added ProcessManager class for global process tracking
  • Implemented _managed_process async context manager
  • Process cleanup with escalation: SIGTERM → SIGKILL → process group termination
  • Prevents zombie processes and ensures resource cleanup
  • Fixed StreamReader close issues and parameter compatibility

3. ✅ Memory-Safe Streaming for Large Files

File: ocode_python/tools/file_tools.py

  • Added offset/limit parameters for streaming large files
  • Prevents loading entire large files into memory
  • Size validation with 50MB default limit
  • Supports partial file reads for efficient processing

4. ✅ Input Sanitization for Shell Commands

File: ocode_python/utils/command_sanitizer.py

  • Comprehensive command validation against dangerous patterns
  • Blocks rm -rf /, fork bombs, and other destructive commands
  • Per-command validation for restricted commands (chmod, chown, etc.)
  • Environment variable sanitization
  • Command injection prevention
  • Integrated into: BashTool replacing old validation methods

5. ✅ Enhanced Encoding Detection

File: ocode_python/tools/file_tools.py

  • Fallback encoding chain: requested → utf-8 → utf-8-sig → latin-1 → cp1252
  • Graceful handling of encoding errors with replacement characters
  • Reports actual encoding used in metadata

6. ✅ Timeout Handling Improvements

File: ocode_python/utils/timeout_handler.py

  • Created comprehensive timeout utilities:
    • async_timeout: Context manager for async operations
    • sync_timeout: Decorator for synchronous functions
    • with_timeout: Utility function with default values
    • TimeoutManager: Cascading timeouts for complex operations
    • AdaptiveTimeout: Dynamic timeout adjustment based on history
  • Enhanced TimeoutError with operation context
  • Integrated into:
    • FileReadTool: Adaptive timeout based on file size
    • MCPTool: Timeout for server connections and tool calls
    • ProcessMonitorTool: Timeout for process queries
    • AgentTool: Timeout for task execution

Completed Improvements (10/10 Tasks) ✅

7. ✅ Create Retry Decorator with Exponential Backoff

File: ocode_python/utils/retry_handler.py

  • Comprehensive retry logic with exponential backoff and jitter
  • Support for both sync and async functions via decorators
  • Configurable retry policies with predefined configurations
  • Integrated into: CurlTool and MCPTool for network resilience
  • Utility functions with_retry() and with_retry_async() for ad-hoc retries
  • Comprehensive test coverage in tests/unit/test_retry_handler.py

8. ✅ Implement Size Validation for JSON/YAML

File: ocode_python/utils/safe_parser.py

  • Size validation for JSON/YAML files to prevent OOM attacks
  • Safe parsing functions with configurable size limits (50MB JSON, 10MB YAML)
  • Streaming support for JSON Lines and multi-document YAML
  • Custom exception classes FileSizeError and ParseError for better error handling
  • Convenience functions for backward compatibility
  • Comprehensive test coverage in tests/unit/test_safe_parser.py

9. ✅ Create Structured Error Classes

File: ocode_python/utils/structured_errors.py

  • Hierarchical error classes with rich context and debugging information
  • Error categories (validation, permission, network, file_system, etc.) and severity levels
  • ErrorContext class for operation details and user data tracking
  • Automatic mapping from standard Python exceptions to structured errors
  • User-friendly error formatting with suggestions and troubleshooting
  • Comprehensive test coverage in tests/unit/test_structured_errors.py

10. ✅ Implement File Operation Retries

File: ocode_python/utils/file_operations.py

  • Resilient file operations with retry logic for transient failures
  • Safe functions for read, write, copy, move, delete operations with atomic writes
  • Async versions of file operations for non-blocking I/O
  • File lock detection and waiting utilities (Windows-specific)
  • Integration with structured errors for consistent error handling
  • Comprehensive test coverage in tests/unit/test_file_operations.py

All Reliability Improvements Complete! 🎉

All 10 reliability improvements have been successfully implemented, providing comprehensive protection against:

  • Path traversal attacks and dangerous file operations
  • Process leaks and resource cleanup issues
  • Memory exhaustion from large files
  • Command injection and shell vulnerabilities
  • Encoding detection and fallback handling
  • Network timeouts and API failures
  • Retry logic for transient failures
  • Size validation for JSON/YAML parsing
  • Structured error handling with rich context
  • File operation failures and locks

Testing

All improvements include comprehensive test coverage:

  • tests/unit/test_path_validator.py
  • tests/unit/test_command_sanitizer.py
  • tests/unit/test_timeout_handler.py
  • tests/unit/test_file_tools_timeout.py
  • Updated existing tool tests

Benefits

  1. Security: Protection against path traversal, command injection, and malicious inputs
  2. Reliability: Proper resource cleanup, timeout handling, and error recovery
  3. Performance: Memory-safe operations for large files, adaptive timeouts
  4. Maintainability: Centralized utilities reduce code duplication
  5. User Experience: Better error messages and graceful degradation

Usage Examples

Path Validation

from ocode_python.utils import path_validator

is_valid, error_msg, validated_path = path_validator.validate_path(
    user_input_path,
    base_path="/safe/directory",
    allow_symlinks=False
)

Command Sanitization

from ocode_python.utils.command_sanitizer import CommandSanitizer

sanitizer = CommandSanitizer()
is_safe, error_msg, safe_cmd = sanitizer.sanitize_command(user_command)

Timeout Handling

from ocode_python.utils.timeout_handler import async_timeout, with_timeout

# Context manager
async with async_timeout(30.0, "file_operation"):
    await process_large_file()

# Utility function
result = await with_timeout(
    fetch_data(),
    timeout=10.0,
    operation="api_call",
    default=[]  # Return empty list on timeout
)