Developer Tools (`tools/`)

The root-level tools/ directory contains developer-facing CLI utilities for database synchronization, S3 media sync, and NFusion product catalog imports. These scripts are run locally (in WSL on Windows) and are never web-accessible.

WSL Required

All bash scripts in tools/ must be run from WSL (Windows Subsystem for Linux) in the project root directory — not from PowerShell or CMD.

cd /mnt/z/Herd/scottsdalemint

Directory Structure

tools/
├── db-sync/
│   ├── db-sync.sh              # Main database synchronization script
│   ├── setup-db-sync.sh        # One-time setup helper
│   ├── close-tunnels.ps1       # (Legacy) PowerShell tunnel closer — no longer needed
│   ├── db-sync.conf.example    # Example configuration file
│   ├── db-sync.conf            # Your local config (git-ignored)
│   └── .env                    # DB credentials (git-ignored)
├── nfusion-import/
│   ├── retail-import.php       # Convert WooCommerce export → NFusion retail catalog CSV
│   ├── retail-tiers-import.php # Convert WooCommerce export → NFusion retail price tiers CSV
│   ├── dealers-import.php      # Convert WooCommerce export → NFusion dealers catalog CSV
│   └── dealers-tiers-import.php# Convert WooCommerce export → NFusion dealers price tiers CSV
├── s3-sync/
│   ├── s3-sync.sh              # S3 uploads synchronization script
│   ├── s3-sync.conf.example    # Example configuration file
│   ├── s3-sync.conf            # Your local config (git-ignored)
│   └── .env                    # AWS credentials (git-ignored)
├── hosts-count.php             # Analyzes hosts.csv to find top 500 hosts by frequency
├── hosts.csv                   # Input CSV for hosts analysis
└── hosts.txt / hosts-results.txt # Input/output for host analysis

Database Sync (`tools/db-sync/`)

Version: 1.1.0
Script: tools/db-sync/db-sync.sh

Synchronizes the WordPress MySQL database between environments (production, staging, development, local). Supports remote-to-remote sync, data scrubbing, URL replacement, and selective operation skipping.

Quick Start

PowerShell / CMD will not work

db-sync.sh is a bash script and must be run inside WSL. Opening it in PowerShell or Windows CMD will fail. Launch a WSL terminal first:

wsl
cd /mnt/z/Herd/scottsdalemint   # adjust drive letter to match your setup

# From the project root in WSL:

# 1. Copy and configure the operational settings file
cp tools/db-sync/db-sync.conf.example tools/db-sync/db-sync.conf
# Edit db-sync.conf as needed (notification email, scrub defaults, etc.)

# 2. Create the credentials .env file (see Configuration section below)
touch tools/db-sync/.env
# Populate it with DB passwords, SSH config, and Aurora endpoints

# 3. Sync production → local (most common use case)
./tools/db-sync/db-sync.sh --source=production --target=local

# 4. Dry-run first to preview operations
./tools/db-sync/db-sync.sh --source=production --target=local --dry-run

Configuration

The db-sync tool uses two separate files that must both exist before running:

File	Purpose	Tracked in git?
`tools/db-sync/db-sync.conf`	Operational settings (copy from `.conf.example`)	No — git-ignored
`tools/db-sync/.env`	Secrets: passwords, SSH keys, Aurora endpoints	No — git-ignored

Creating `tools/db-sync/.env`

Create this file manually — there is no .env.example provided since it contains secrets. Populate it with the following variables (get values from a team member or the Ymir/AWS console):

################################################################################
# Database Sync Configuration
# KEEP THIS FILE SECURE AND DO NOT COMMIT TO VERSION CONTROL
################################################################################

# Database Passwords
DB_PASS_LIVE=<production_aurora_password>
DB_PASS_STAGING_DEVELOPMENT=<staging_and_dev_aurora_password>

# SSH Bastion Host Configuration
# These bastion hosts provide access to the Aurora databases
SSH_BASTION_STG_DEV="bastion-stg-dev.scottsdalemint.com"
SSH_BASTION_PROD="bastion-prod.scottsdalemint.com"
SSH_USER="ubuntu"
SSH_KEY_PATH="/home/<your_wsl_username>/.ssh/scottsdale_bastion_key"

# Aurora Database Endpoints
# Actual database server addresses accessed through the bastion hosts
DB_HOST_STG_DEV="scottsdalemint-aurora-dev.cluster-xxxxx.us-east-1.rds.amazonaws.com"
DB_HOST_PROD="scottsdalemint-aurora-prod.cluster-xxxxx.us-east-1.rds.amazonaws.com"

# Local Database Configuration
# Run 'ipconfig' in cmd.exe to find your local machine IP address
LOCAL_DB_NAME="wp_scottsdale"
LOCAL_DB_HOST="<your_local_machine_ip>"

Finding your local IP

Run ipconfig in Windows CMD and use the IPv4 address of your active network adapter (e.g., 10.10.15.x). This is needed so WSL can reach your local MySQL instance.

Getting the SSH key

The scottsdale_bastion_key private key is required to tunnel through the bastion hosts to Aurora. Obtain this from a senior developer or the shared secrets vault and place it at the path specified in SSH_KEY_PATH. Ensure permissions are set correctly:

chmod 600 ~/.ssh/scottsdale_bastion_key

Supported Environments

Value	Description
`production`	Live site (read-only — export only, never a target)
`staging`	Staging environment on AWS
`development`	Dev environment on AWS
`local`	Local Herd instance (import only, never a source)

Key Features

Feature	Description
Multi-environment sync	Export from prod/staging/dev, import to staging/dev/local
Remote-to-remote sync	Directly sync staging ↔ development without pulling locally
Data scrubbing	Optionally delete orders (60+ days old) and user data on local targets
URL replacement	Automatic `search-replace` for environment URLs
Dry-run mode	Preview all operations without making changes
Lockfile protection	Prevents accidental parallel execution
Email notifications	SMTP alerts on completion or failure
Auto WP-CLI install	Installs WP-CLI globally if not found

Common Flags

--source=<env>        Source environment (production, staging, development)
--target=<env>        Target environment (staging, development, local)
--dry-run             Preview operations without executing them
--skip-migration      Skip running WP database migration after import
--skip-search-replace Skip URL search-replace step
--skip-scrub-orders   Keep all orders (skip order deletion scrub)
--skip-scrub-users    Keep all users (skip user deletion scrub)

How It Works

Opens an SSH tunnel via Ymir CLI to the Aurora database.
Exports the source database with mysqldump.
Runs data-scrubbing SQL to remove orders/users (local targets only, optional).
Imports the dump into the target database.
Runs wp search-replace to swap domain URLs.
For remote targets, uses ymir wp <environment> instead of local WP-CLI.

Requirements

Bash 4.0+ or WSL
MySQL client and mysqldump
PHP 7.4+
curl
Ymir CLI (vendor/bin/ymir)
WP-CLI (auto-installed if absent)
sendemail (optional, for notifications)

S3 Media Sync (`tools/s3-sync/`)

Version: 1.0.0
Script: tools/s3-sync/s3-sync.sh

Syncs the WordPress uploads/ folder between S3 buckets (production, staging, development) and the local filesystem. Uses parallel processing for faster transfers.

Quick Start

# Copy and configure credentials
cp tools/s3-sync/s3-sync.conf.example tools/s3-sync/s3-sync.conf
# Add AWS credentials to tools/s3-sync/.env

# Sync production S3 → local uploads folder
./tools/s3-sync/s3-sync.sh --source=production --target=local

# Dry-run first
./tools/s3-sync/s3-sync.sh --source=production --target=local --dry-run

Key Features

Feature	Description
Year-folder filtering	Only syncs `2013–2099` year folders, ignoring cache/temp directories
One-way sync	Files are never deleted from the target (safe append-only)
Parallel processing	Up to 10 year folders synced simultaneously
Dry-run mode	Test without making any changes
Lockfile protection	Prevents parallel execution
Per-year logs	Individual log files for each year folder synced

Requirements

Bash 4.0+ or WSL
AWS CLI with valid credentials
S3 bucket access to Ymir asset buckets

NFusion Import Tools (`tools/nfusion-import/`)

PHP scripts for converting WooCommerce product exports into the CSV format required by the NFusion Solutions product catalog and price-tier import system. Run these when re-syncing the product catalog with NFusion after bulk product changes.

Scripts

`retail-import.php`

Converts a WooCommerce retail product export CSV (retailscottsdale-productos.csv) into nfusion-retail-import.csv for NFusion catalog import.

Field Mappings:

WooCommerce Field	NFusion Field	Notes
`SKU`	`Sku`	Required
`Name`	`Name`	Required
`Meta: _metal_weight`	`Ounces`	4 decimal places
`Meta: _markup_rate` × `Meta: _metal_weight`	`AskDollar`	Markup per oz × total oz
`Meta: _metal_type`	`Metal`	Title-cased (e.g., `Silver`)
Metal type	`ProductFamily`	e.g., `"Silver Bullion"`

ProductFamily mapping:

`_metal_type`	`ProductFamily`
`silver`	`Silver Bullion`
`gold`	`Gold Bullion`
`copper`	`Copper Bullion`
`platinum`	`Platinum Bullion`
`palladium`	`Palladium Bullion`

# Place retailscottsdale-productos.csv in nfusion-import/, then:
php tools/nfusion-import/retail-import.php
# Outputs: tools/nfusion-import/nfusion-retail-import.csv

`retail-tiers-import.php`

Converts volume pricing tiers from the WooCommerce retail export into nfusion-price-tiers-import.csv for NFusion price-tier import.

php tools/nfusion-import/retail-tiers-import.php

`dealers-import.php`

Same as retail-import.php but reads from dealersscottsdale-productos.csv and outputs nfusion-dealers-import.csv for the dealers channel catalog.

php tools/nfusion-import/dealers-import.php

`dealers-tiers-import.php`

Converts dealers volume pricing tiers into nfusion-dealers-tiers-import.csv.

php tools/nfusion-import/dealers-tiers-import.php

Typical Workflow

Export dealer and retail product data from WooCommerce (WP Admin → Products → Export).
Save CSV files to tools/nfusion-import/ with the expected filenames.
Run the appropriate PHP scripts.
Upload the generated CSV files to the NFusion catalog import portal.

Hosts Analysis (`tools/hosts-count.php`)

A standalone PHP utility for analyzing a hosts.csv file and identifying the top 500 hosts by frequency. Used for traffic analysis and CDN/firewall rule creation.

php tools/hosts-count.php
# Reads:  tools/hosts.csv
# Writes: tools/hosts-results.txt

This script is a one-off utility and is not part of any automated process.

Security Notes

tools/db-sync/.env and tools/s3-sync/.env contain database passwords and AWS credentials — they are git-ignored and must never be committed.
Database sync with --target=production is intentionally disabled — production is export-only.
Remote-to-remote sync shows an extra confirmation warning to prevent accidental data overwrites.

Directory Structure​

Database Sync (tools/db-sync/)​

Quick Start​

Configuration​

Creating tools/db-sync/.env​

Supported Environments​

Key Features​

Common Flags​

How It Works​

Requirements​

S3 Media Sync (tools/s3-sync/)​

Quick Start​

Key Features​

Requirements​

NFusion Import Tools (tools/nfusion-import/)​

Scripts​

retail-import.php​

retail-tiers-import.php​

dealers-import.php​

dealers-tiers-import.php​

Typical Workflow​

Hosts Analysis (tools/hosts-count.php)​

Security Notes​

Directory Structure

Database Sync (`tools/db-sync/`)

Quick Start

Configuration

Creating `tools/db-sync/.env`

Supported Environments

Key Features

Common Flags

How It Works

Requirements

S3 Media Sync (`tools/s3-sync/`)

Quick Start

Key Features

Requirements

NFusion Import Tools (`tools/nfusion-import/`)

Scripts

`retail-import.php`

`retail-tiers-import.php`

`dealers-import.php`

`dealers-tiers-import.php`

Typical Workflow

Hosts Analysis (`tools/hosts-count.php`)

Security Notes