Nabu Catalog

June 17, 2026 ยท View on GitHub

Setting up your dev environment

Use direnv to add bin to your path

PATH_add bin

Bring up the environment

# Build the base image
docker compose build

# Install the gems
nabu_run bundle

# Bring up all the containers
docker compose up

This brings up the following containers

  • app - the rails app
  • search - elasticsearch instance for search (dev + test)
  • db - mysql data base (dev + test)
  • s3 - s3 mock

You can then easily run all the standard commands by prefixing with nabu

nabu_run bundle install
nabu_run bundle exec rake db:prepare
nabu_run bundle exec guard # Test runner

Production

The application is designed to be deployed with containers into an AWS account using CDK

To bootstrap a new account

# Setup an AWS account and credentials as per your preferred method and set the environment to use it
AWS_PROFILE=nabu
REGION=ap-southeast-2
ACCOUNT=$(aws sts get-caller-identity | jq -r .Account)
cdk bootstrap aws://$ACCOUNT/$REGION

If ECR complains about access

ACCOUNT=$(AWS_PROFILE=nabu-stage aws sts get-caller-identity | jq -r .Account)
AWS_PROFILE=nabu-stage aws ecr get-login-password --region ap-southeast-2 | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.ap-southeast-2.amazonaws.com

Deployment

Use CDK to deploy new code via docker as well as any infrastructure changes

bin/release stage
bin/release prod

If necessary:

bin/aws/ecs_rake app deploy:migrate
bin/aws/ecs_rake app searchkick:reindex

Importing a production database into your development environment

bin/aws/db_sync dumps the production database to S3, restores it into your local nabu_devel, and resets every user's password to password. See docs/runbooks/db-sync.md for the full design (it can also overwrite staging).

bin/aws/db_sync   # choose target 1) dev
nabu_run bin/rails db:environment:set RAILS_ENV=development
nabu_run bin/rake db:migrate
nabu_run bin/rake searchkick:reindex:all

New Ethnologue data

We use the following source locations

Run the following rake task to import everything

bundle exec rake import:ethnologue

This will

  • Add new countries and update names
  • Update country names
  • Add new languages and update names (Only Living languages)
  • Add mappings of language to countries
  • All existing languages that are retired are marked as such, incl name change.
  • Where name changes occurred items in CollectionLanguage, ItemContentLanguage, ItemSubjectLanguage are updated with the replacement language code.
  • Where splits happened, a message is printed.

OAI-PMH

OLAC available at:

The feeds that OLAC harvests:

Individual item:

RIF-CS available at:

The feed that ANDS harvests:

Test at ANDS:

Feed for a single collection:

To validate our XML output as per OLAC

java -cp xercesImpl.jar:xercesSamples.jar sax.Counter -n -np -v -s -f item.xml

Setup Secrets

aws secretsmanager list-secrets

aws secretsmanager put-secret-value --secret-id ARN --secret-string "{\"site_key\":\"***\", \"secret_key\":\"***\"}"

Upgrades

We should regularly make sure we are running the latest versions of third-party packages

# Ruby gems
nabu_run bundle outdated
nabu_run bundle update

# node modules
nabu_run pnpm up -i

# New rails version
rails new nabu --database=mysql --javascript=esbuild --css=sass --skip-action-cable --skip-kamal