Linux Setup

Norgate Data Updater (NDU) only runs on Windows. On Linux, you have two options:

1. WinBoat (recommended) - Containerized Windows VM with seamless file sharing
2. Separate Windows machine - Run the export on Windows, copy files to Linux

Option 1: WinBoat (Recommended)

WinBoat runs Windows in a container with native desktop integration. Windows apps appear as normal windows, and your Linux home directory is accessible from Windows.

How File Sharing Works

WinBoat exposes your Linux home directory via a network share:

From Windows	Maps to Linux
\\host.lan\Home	$HOME (~/)
\\host.lan\Home\norgate_export	~/norgate_export

Files saved in Windows appear instantly on Linux - no manual copying needed.

Setup Steps

1. Install WinBoat

Follow the WinBoat installation guide.

Requirements:

• Docker or Podman
• KVM acceleration enabled
• Windows license

2. Create Export Directory (on Linux)

mkdir -p ~/norgate_export

3. Install NDU in WinBoat

1. Launch WinBoat
2. Download Norgate Data Updater
3. Install NDU and sign in with your Norgate account
4. Let NDU download all historical data (first sync takes hours)

4. Install Python in WinBoat

# Install Python (in Windows)
winget install Python.Python.3.11

# Install required packages
pip install norgatedata pandas

5. Map Network Drive (Optional)

For easier access, map the Linux folder as a drive letter:

net use N: \\host.lan\Home\norgate_export /persistent:yes

Now N:\ points to ~/norgate_export on Linux.

6. Clone This Repository (in WinBoat)

# Navigate to the network share
cd \\host.lan\Home

# Clone the repository
git clone https://github.com/davidszp/norgate-pst-utils.git

7. Run the Export

cd \\host.lan\Home\norgate-pst-utils

# Make sure NDU is running (check system tray)

# Export all mapped instruments
python norgate_utils/export.py --output-dir N:\

# Or with UNC path
python norgate_utils/export.py --output-dir "\\host.lan\Home\norgate_export"

8. Import to PST (on Linux)

Files are already on Linux - no copy step needed!

# Verify files exist
ls ~/norgate_export/
# CRUDE_W_20230300.csv, GOLD_20240200.csv, SP500_20241200.csv, ...

# Continue with PST import
cd /path/to/pysystemtrade
python sysinit/futures/contract_prices_from_csv_to_db.py

Workflow Diagram

LINUX HOST
===========

  ~/norgate_export/  <---------- Files appear here automatically
        |
        | (shared filesystem)
        |
        v
  +---------------------------------------------+
  | WINBOAT (Windows Container)                 |
  |                                             |
  |   NDU (Norgate Data Updater)                |
  |     - Downloads futures data from Norgate   |
  |                                             |
  |   Python + norgatedata                      |
  |     - Exports to \\host.lan\Home\...        |
  +---------------------------------------------+
        |
        | (same files, no copy needed)
        v
  pysystemtrade
    - Imports from ~/norgate_export/

Troubleshooting WinBoat

host.lan Permission Errors

If you get "Access Denied" when writing to \\host.lan\Home:

Fix 1: Disable host Samba service (it may conflict)

sudo systemctl stop smbd
sudo systemctl disable smbd

Fix 2: Check WinBoat's Samba configuration - see GitHub issues

host.lan Not Accessible

If \\host.lan doesn't appear in Windows Network:

1. Ensure WinBoat is running with network sharing enabled
2. Try direct access: Win+R → \\host.lan\Home
3. Restart WinBoat

Option 2: Separate Windows Machine

If WinBoat doesn't work for you, use a separate Windows machine (physical or VM):

1. On Windows

Follow the Windows Setup Guide to run the export.

2. Transfer Files to Linux

# Option A: SCP/SFTP
scp -r user@windows-machine:/c/norgate_export/ ~/norgate_export/

# Option B: rsync over SSH
rsync -avz user@windows-machine:/c/norgate_export/ ~/norgate_export/

# Option C: Shared network drive
cp -r /mnt/windows_share/norgate_export/ ~/norgate_export/

# Option D: Tar and transfer
# On Windows: tar -czvf norgate_export.tar.gz norgate_export/
# Transfer the tar.gz, then:
tar -xzvf norgate_export.tar.gz

3. Import to PST

cd /path/to/pysystemtrade

# Copy files to PST data directory
cp -r ~/norgate_export/ data/futures/norgate_export/

# Run PST import scripts
python sysinit/futures/contract_prices_from_csv_to_db.py

Performance Notes

• WinBoat uses KVM acceleration - performance is comparable to native Windows
• Network share may be slower than native disk for large exports
• First export of ~23,000 contracts takes 30-60 minutes regardless of method