India-Forecast-App

⚠️ WARNING: We are in the process of deprecating this app and moving to the site-forecast-app this repo will be archived soon

Runs wind and PV forecasts for India and saves to database

Install dependencies (requires poetry)

poetry install

Linting and formatting

Lint with:

make lint

Format code with:

make format

Running tests

make test

⚠️ Note: one test for the AD model is skipped locally unless the HF token is set, this HF token can be found in AWS Secret Manager under {environment}/huggingface/token and then can be set via export HUGGINGFACE_TOKEN={token_value} in the repo to run the additional test. In CI tests this secret is set so the test will run there.

Running the app locally

Replace {DB_URL} with a postgres DB connection string (see below for setting up a ephemeral local DB)

If testing on a local DB, you may use the following script to seed the the DB with a dummy user, site and site_group.

DB_URL={DB_URL} poetry run seeder

⚠️ Note this is a destructive script and will drop all tables before recreating them to ensure a clean slate. DO NOT RUN IN PRODUCTION ENVIRONMENTS

This example runs the application and writes the results to stdout

DB_URL={DB_URL} NWP_ZARR_PATH={NWP_ZARR_PATH} poetry run app

To save batches, you need to set the SAVE_BATCHES_DIR environment variable to directory.

Saving to OCF Data Platform

To optionally save forecasts to the OCF Data Platform via gRPC, configure the following environment variables:

• SAVE_TO_DATA_PLATFORM (set to "True" to enable, defaults to "False")
• DATA_PLATFORM_HOST (defaults to "localhost")
• DATA_PLATFORM_PORT (defaults to "50051")

To optionally read input generation data from the Data Platform (instead of the pvsite database) via gRPC, configure:

• READ_FROM_DATA_PLATFORM (set to "True" to enable, defaults to "False")
• OBSERVER_NAME (the DP observer to read observations from, defaults to "india")
• DATA_PLATFORM_HOST (defaults to "localhost")
• DATA_PLATFORM_PORT (defaults to "50051")

Starting a local database using docker

    docker run \
        -it --rm \
        -e POSTGRES_USER=postgres \
        -e POSTGRES_PASSWORD=postgres \
        -p 54545:5432 postgres:14-alpine \
        postgres

The corresponding DB_URL will be

postgresql://postgres:postgres@localhost:54545/postgres

Building and running in Docker

Build the Docker image

make docker.build

Create a container from the image. This example runs the application and writes the results to stdout.
Replace {DB_URL} with a postgres DB connection string.
N.B if the database host is localhost on the host machine, replace localhost with host.docker.internal so that docker can access the database from within the container

docker run -it --rm -e DB_URL={DB_URL} -e NWP_ZARR_PATH={NWP_ZARR_PATH} ocf/india-forecast-app

Notes

This repo makes use of PyTorch (torch and torchvision packages) CPU-only version. In order to support installing PyTorch via poetry for various environments, we specify the exact wheels for each environment in the pyproject.toml file. Some background reading on why this is required can be found here: https://santiagovelez.substack.com/p/how-to-install-torch-cpu-in-poetry?utm_campaign=post&utm_medium=web&triedRedirect=true

Contributors ✨

Thanks goes to these wonderful people (emoji key):

priyanshubajaj ⚠️

Peter Dudfield 💻

Dakshbir 📖

MAYANK SHARMA 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

Troubleshooting

Poetry Installation Issues

Problem: poetry install fails with dependency conflicts Solution: Try updating Poetry first with pip install --upgrade poetry, then run poetry update followed by poetry install

Problem: Package installation errors Solution: Check your Python version matches the one specified in pyproject.toml. You can use poetry env use python3.x to set the correct version.

Docker Database Connection Issues

Problem: Container can't connect to local database with "connection refused" error Solution: If using localhost in your DB_URL, replace it with host.docker.internal when running in Docker

Problem: Database authentication failures Solution: Verify your DB_URL format is correct: postgresql://username:password@hostname:port/database

Model Loading Issues

Problem: "Failed to load model" errors Solution: Ensure your HUGGINGFACE_TOKEN environment variable is set correctly. The token can be found in AWS Secret Manager under {environment}/huggingface/token.

Problem: Out of memory errors when loading models Solution: Ensure your system has sufficient RAM, or consider using a smaller model variant.